Macro and Micro Level Challenges to Programevaluation and Accountability

The demand for documenting outcomes of extension programs continues to increase. This study examined challenges to program evaluation and accountability. The challenges were categorized into two levels--macro level (national) and micro level (grass roots or state level). The findings indicated several challenges--five macro and six micro level challenges. Based on the review and personal experience of the author, a framework to address the challenges was developed. The framework consisted of three components--using accountability information, customer satisfaction, and developing a generic evaluation instrument to assess program impact. In addition, three strategies--communication, teamwork and distance education were suggested to build evaluation capacity among county staff. Introduction In the past Extension programs focused on convincing clientele to follow certain practices such as planting hybrid corn, preserving food safely, etc. An Extension agent was viewed as a change agent (Sawer, 1984). The concept of "innovators" "early adopters" became a focal point of a program of research. Attempts were made to evaluate Extension programs using the "educational objective" approach of the formal school system. The concept of "program objectives" then came into existence. And following that the focus of extension programs shifted more toward knowledge, attitudes/opinions, skills, and aspirations (KA(O)SA) change (Bennett, 1979). Extension agents were viewed as educators (Sawyer, 1984). During this time, large amounts of money were allocated to social intervention programs which aimed at eliminating poverty and improving health care. Extension programs were also expanded to reaching urban and other non-traditional audiences. Evaluation of these programs tended to be of informal judgment rather than formal assessment. Systematic measurement of outcomes was elusive and programmers used testimonials, case studies, happiness indexes etc., to show the worth of their programs. The expected outcomes were hard to document using these methods (Patton, 1978; Smith, 1981; and Sawer, 1984, Boyle, 1996). Several federal statutes were enacted which demanded greater accountability. Examples of legislation included: 1) Federal Regulations for the Community Mental Health Centers Act of 1964, which required needs assessment; 2) the Social Security Amendment Act of 1972, which established reviews for checking appropriateness and quality of services provided; 3) Federal Regulations for the Community Mental Health Center Amendment Act of 1975, which authorized for funds to do evaluation; 4) the Farm Bill of 1977, which mandated large-scale evaluations of the social and economic consequences of Extension programs; and 5) the Sunset Act of 1978, which combined concepts of program review and zero-based budgeting. These legislations made clear that it was no longer enough to report dollars spent on what and how many participated etc. This lead to competition of funds and greater demands for program effectiveness and efficiency. Number of programs offered, participants reached, hours worked and dollars spent were no longer adequate to assess program effectiveness. Accountability is not new to extension. However, its importance has become more pronounced because of stricter mandates from federal, state, local and university level legislation. According to Ladewig (1997), Cooperative Extension, like all public agencies, has seen an increased emphasis from government on program performance and accountability. The role of accountability in the budget process is certainly on the rise (Irwin, 1999) and Cooperative Extension is no exception. Accountability is defined as an implied or explicit requirement to accept responsibility for performance, progress, accomplishment, effectiveness or success of a program, activity, or project in terms of results achieved (South Carolina State Government Quality Network Association, 1998). Accountability requirements and reporting systems at the federal level have gone through a variety of different approaches throughout the history of Extension from the Extension Management Information System (EMIS) in 1970 to the National Accomplishments Reporting System (NARS) in 1982 to the Program Planning and Reporting System (PPARS) in 1992 to Government Performance Results Act (GPRA) in 1993. These frequent changes in reporting systems in Extension indicate the complexity of the reporting systems, burden on states to comply with new systems of reporting, and challenging states to develop their own reporting systems to meet state, federal and local mandates. Almost all land-grant universities, including 1890s and 1994s, have some method of reporting-DOS, electronic, Web-based, and other manual means of collecting information to comply with state and federal reporting requirements. Furthermore, several studies have identified lack of time, lack of resources, and limited expertise in evaluation methodology, in developing surveys, data analysis and reporting, as factors inhibiting agents to conduct systematic evaluation of their programs (Kiernan, Fennelly, Mulkeen, Mincemoyer, Cornell, Masters, Radhakrishna, Lewis, and Baggett, 1994; Ott, 1996; and Depp, 1996).